On the Bayesian “occam Factors” Argument for Occam’s Razor

نویسنده

  • David H. Wolpert
چکیده

This paper discusses some of the problematic aspects of the Bayesian first-principles “proof” of Occam’s razor which involves Occam factors. Although it is true that the posterior for a model is reduced due to Occam factors if that model is capable of expressing many functions, the phenomenon need not have anything to do with Occam’s razor. This paper shows this by i) performing reductio ad absurdum on the argument that the Occam factors effect implies Occam’s razor; ii) presenting an alternative Bayesian approach which explicitly does not result in Occam’s razor; and finally iii) disentangling the underlying problem with viewing the Occam factors argument as a proof or “automatic embodiment” of Occam’s razor.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Solomonoff Prediction and Occam’s Razor

Algorithmic information theory gives an idealized notion of compressibility, that is often presented as an objective measure of simplicity. It is suggested at times that Solomonoff prediction, or algorithmic information theory in a predictive setting, can deliver an argument to justify Occam’s razor. This paper explicates the relevant argument, and, by converting it into a Bayesian framework, r...

متن کامل

Occam's Razor

The Bayesian paradigm apparently only sometimes gives rise to Occam’s Razor; at other times very large models perform well. We give simple examples of both kinds of behaviour. The two views are reconciled when measuring complexity of functions, rather than of the machinery used to implement them. We analyze the complexity of functions for some linear in the parameter models that are equivalent ...

متن کامل

How Occam's Razor Provides a Neat Definition of Direct Causation

In this paper we show that the application of Occam’s razor to the theory of causal Bayes nets gives us a neat definition of direct causation. In particular we show that Occam’s razor implies Woodward’s (2003) definition of direct causation, provided suitable intervention variables exist and the causal Markov condition (CMC) is satisfied. We also show how Occam’s razor can account for direct ca...

متن کامل

Occam's Razor and a Non-Syntactic Measure of Decision Tree Complexity

Occam's razor, attributed to the fourteenth century English philosopher William of Occam, states: “plurality should not be assumed without necessity.” The machine learning interpretation of Occam’s razor is that if two models have the same performance on the training set, choose the simpler. Decision tree learning widely uses Occam’s razor. Popular decision tree generating algorithms are based ...

متن کامل

A Syntactic Justification for Occam’s razor

Informally, Occam’s razor states, “Given two hypotheses which equally agree with the observed data, choose the simpler”, and has become a central guiding heuristic in the empirical sciences and in particular machine learning. We criticize previous arguments for the validity of Occam’s razor. The nature of hypotheses spaces is explored and we observe a correlation between the complexity of a con...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 1992